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Abstract 

This study investigates the relationship between approaches to studying and 
course completion in two online preparatory university courses in math¬ 
ematics and computer programming. The students participating in the two 
courses are alike in age, gender, and approaches to learning. Four hundred 
and ninety-three students participating in these courses answered the short 
version of the Approaches and Study Skills Inventory for Students (ASSIST). 
Results show that students demonstrating a deep approach to learning in ei¬ 
ther course are more likely to complete. In the mathematics course, a com¬ 
bination of deep and strategic approaches correlates positively with course 
completion. In the programming course, students who demonstrate a surface 
approach are less likely to complete. These results are in line with the inten¬ 
tions of the course designers, but they also suggest ways to improve these 
courses. Furthermore, the study demonstrates that ASSIST can be used to 
evaluate course design. 
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Resume 

Cette recherche examine la relation entre les differentes approches envers les 
etudes et la reussite des cours adoptees par les etudiants de deux differents 
cours universitaires offerts en ligne : l’un en mathematiques, l’autre en 
programmation. Dans les deux cours, les participants presentent les memes 
caracteristiques quant a l’age, au sexe et a l’attitude vis-a-vis des etudes. 
Quatre cent quatre-vingt-treize etudiants inscrits a ces cours ont repondu a la 
version abregee dune recherche sur les approches et les competences d’etudes 
chez les etudiants intitulee « Approaches and Study Skills Inventory for 
Students » (ASSIST). Les resultats demontrent que les etudiants qui abordent 
l’apprentissage de maniere serieuse dans l’un ou l’autre des cours ont plus 
de chances de reussir. Dans le cours de mathematiques, une combinaison 
d’approches serieuses et strategiques est positive pour reussir le cours, tandis 
que les etudiants du cours de programmation qui adoptent une approche 
superficielle envers l’apprentissage presentent moins de chance de reussite. 
Bien que ces resultats soient conformes aux intentions des createurs de ces 
cours, ils proposent aussi certaines ameliorations possibles. L’etude demontre 
qu’on peut aussi utiliser ASSIST pour 1 ’evaluation de la conception de cours. 


Course completion is an outcome of interest in online courses, as the dropout rate is 
often higher when compared with on-campus courses (Levy, 2007). In this study we in¬ 
vestigate two preparatory university courses offered online, with variation in completion 
rates: 37% in a mathematics course and 69% in a programming course. The courses target 
the same student group, and on-campus equivalents have completion rates over 90%. The 
aim of this study is to explore whether the Approaches and Study Skills Inventory for Stu¬ 
dents (ASSIST) can be used as an effective instrument to evaluate students in an online 
course and, if so, whether students’ approaches to learning can explain the difference in 
course completion. 

Background Information 
Approaches to Learning and Studying 

The approaches to learning theory derives from qualitative analysis of in-depth inter¬ 
views with students who describe their learning behaviour and intentions when studying 
a text (Marton & Saljo, 2005). From students’ focus on extracting meaning from this text 
but with different emphasis on outcome and process, two distinctive approaches were 
identified: deep-level and surface-level processing. The difference between them could be 
explained by a passive or active attitude to learning (Dahlgren & Marton, 1978). Distinc¬ 
tions are made according to the references students make about intention and organi¬ 
zation, where motivation increases deep-level processing and anxiety increases surface- 
level processing (Entwistle, 1977). 

A deep approach to learning is characterized by the student’s will and intention to real¬ 
ize an understanding of the study material in order to relate it to a larger context. A surface 
approach is characterized by a focus on mechanistic repetition to memorize information 
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instead of understanding it, in order to automatically reproduce it (Marton & Saljo, 2005). 
Subsequent research added a third and complementary category: a strategic approach to 
studying. This is an achieving orientation, in which the student’s ambition is to organize 
learning in an effective way to fulfill course requirements (Tait & Entwistle, 1996). 

The three approaches can be divided into different subscales, in order to further detail 
aspects of studying. In the short version used here, each approach has three subscales 
that express variants of motive and strategy. For the deep approach, the subscales are 
(1) seeking meaning, (2) relating ideas, and (3) use of evidence, all of which are driven 
by a meaning orientation and deep motivation. The strategic approach subscales are (1) 
time management, (2) achieving, and (3) organized studying. The surface approach is 
divided into (1) unrelated memorizing, (2) lack of purpose, and (3) fear of failure. This 
latter group comprises students who have a more extrinsic motivation with a reproducing 
orientation (Baeten, Kyndt, Struyven, & Dochy, 2010). 

There is also evidence that students’ approaches to learning are driven by their moti¬ 
vation and attitude; that is, “by the extent they are able to adopt congenial approaches to 
studying” (Richardson, 2007). These approaches are not contradictory; all students are 
believed to have some aspects of all three. Each may emerge in different combinations 
based on the context (Diseth, 2003). 

Academic Achievement 

Approaches to learning have been reported to have an impact on the outcome of learn¬ 
ing; high academic achievement has been positively related to a deep or strategic ap¬ 
proach and negatively related to a surface approach (Diseth & Martinsen, 2003; Diseth 
2003). This relationship occurs provided that the assessment requires high levels of un¬ 
derstanding from the students (Anderson, Lee, Simpson, & Stein, 2011). It is argued that 
this relationship is typically found among graduate students, whereas undergraduate sci¬ 
ence students may be more inclined to adopt a combination of strategic and surface ap¬ 
proaches (Entwistle, Tait, & McCune, 2000). 

Alternatively, studies among undergraduate psychology and medical students have 
found that a deep approach did not predict academic achievement; rather, a strategic 
approach is the best predictor in these disciplines (Diseth & Martinsen, 2003; Newbie & 
Hejka, 1991). These findings are attributed to the learning environment: one field has a 
strictly fixed and defined curriculum, and the other has an overloaded curriculum, which 
counteracts a deep approach to learning. The pressure on students to achieve is also men¬ 
tioned as a possible explanation, related to the demand for high examination grades in 
these specific subject areas. 

Higher education curriculum often relies on the assumption that students arrive with 
an understanding of basic concepts and principles in core subjects (Dahlgren & Marton, 
1978). To be able to adopt a deep approach to learning, adequate prerequisites are need¬ 
ed; a poor background in the subject or the failure to use previous knowledge while learn¬ 
ing will detract from using a deep approach (Entwistle, McCune, & Hounsell, 2002). 

The approaches to learning theory has been found to relate to pass rates and grades in 
an online environment, but no relation has been found to course completion (Anderson 
et al., 2011). Lower scores on the surface approach and higher scores on achievement mo¬ 
tivation are both related to getting a good grade (Richardson, Morgan, & Woodley, 1999). 
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Online Course Retention 

The retention rate in online courses ranges in general between 60% and 75% (Levy, 
2007). In Sweden, where this study was conducted, it is as low as 56%, as compared with 
an average of 79% for all courses in higher education (HSV, 2011). The Swedish National 
Agency for Higher Education (HSV, 2011) reports that one explanation may be that online 
courses are generally given as individual courses. The retention rate in online individual 
courses is 47%, while in online program courses it is 80%. The retention rate for program 
courses given online is roughly the same as for the equivalent given on campus, but indi¬ 
vidual courses given online have about 20% higher retention rates than the correspond¬ 
ing courses given on campus. 

Tinto’s (1987) theory of student departure emphasizes academic and social integra¬ 
tion of students, where both students and universities bear the responsibility. This is a 
dominant conceptual framework in the field of retention in higher education (Hatos & 
Suta, 2011; Melguizo, 2011). Critics of the theory claim that the empirical support for 
the theory is weak and point out a lack of attention to the influence of external factors 
and to demographic, technological, and institutional changes. Melguizo (2011) points out 
that most of the research in the field of retention is quantitative, and she recommends a 
more qualitative and interdisciplinary approach in order to address the issue of college 
persistence and attainment. However, in self-paced, independent online courses, where 
interaction with institutional members and other students is limited, the model may be 
hard to apply. 

Major factors in students’ decisions to complete an online course are satisfaction with 
the online experience (Hatos & Suta, 2011) and computer/technological literacy required 
to handle the technology used (Dupin-Bryant, 2004). Less experienced students in an 
early term of their academic studies are more likely to drop out (Levy, 2007). This deci¬ 
sion is related more often to lack of persistence than to knowledge gains: “Persistence as a 
phenomenon characterizes the constellation of behaviours, attitudes, skills needed by the 
student to successfully complete an online course” (Hart, 2012, p. 39). 

Method 


Participants 

An invitation letter was sent via email to all students who signed up for one of the two 
courses (mathematics and programming) and who acquired an account for at least one of 
them in June and July 2010. The letter informed the students of the purpose of the study 
and the volunteer participation. A few hours later another email was sent containing both 
a link to the web questionnaire and a link that registered the student as declining partici¬ 
pation in the study. Students not selecting either of these options were sent reminders 
one, two, and three weeks later. 

There were 2,075 invitations distributed. Out of these, 420 students declined partici¬ 
pation. From the remaining 1,655 students, 493 students responded, for a response rate 
of 30%. According to Dillman et al. (2009) and Cook, Heath, and Thompson (2000), this 
response rate is in the range of what can be expected for large-sample surveys similar to 
this study. 
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Courses 

Participants were sampled from two preparatory courses in mathematics and pro¬ 
gramming, both delivered online with self-paced learning. The courses are held during 
the whole year, but a large proportion of participants attend during the summer. These 
courses are designed to even out the variations in levels of knowledge among new stu¬ 
dents applying to university programs containing mathematics in Sweden. To quality for 
participation in these courses, students must pass at least the third level of five levels in 
mathematics as part of the core secondary school curriculum. This level is also a require¬ 
ment for admission; there are no requirements for prior knowledge in computer science 
or programming. The target students are the same for the two courses, and they are all 
invited to both courses. 

All students have access to mentors who are available by either phone or email to assist 
with any questions that may arise during the course. There is also a course forum where 
students have the opportunity to engage in discussion with their peers and the mentors. 

The preparatory course in mathematics, which provides five European Credit Transfer 
and Accumulation System (ECTS) credits, gives the students an advanced repetition of 
secondary-level mathematics required for studies in higher education. It is divided into 
five subsequent parts, each consisting of related exercises with a basic test and a final test. 
For each of the final tests, the student has to solve a problem, first individually and then in 
a group of three other students. The programming course (three ECTS credits) introduces 
some basic elements of programming and computer logic thinking. The course consists of 
four different exercises embedded in the course material to give a basic understanding of, 
for example, how a file system is constructed. 

Both courses use computer assessment, mostly generic questions (Balter, Enstrom, 
& Klingenberg, 2013), as well as multiple-choice and fill-in the blanks questions. If the 
student gives an incorrect response, they receive feedback immediately after the test is 
completed, with a reference about where to read more about the subject. This is designed 
to promote high levels of scholastic attitude and learning outcomes, and the premise that 
immediate feedback is related to student motivation (van der Kleij, Eggen, Timmers, & 
Veldkamp, 2012). The requirement for performance and completion is simple: when a 
student has successfully completed all the assessments during the course, they pass the 
course. Course completion is equal to passing the course; the only grades are pass or fail. 

Dropout Students 

By the Swedish definition of who counts as a course participant (i.e., a student who 
will render revenue to the university), a student has to be active during the first three 
weeks of a course. This study focuses on the courses during the summer semester. 

Students who have dropped out give the following reasons: they changed their minds 
about participating in the course; they chose another course; they were able to get a job 
during the summer; or they signed up for courses only in order to get financial aid for 
studies and never planned to attend the course at all. Few students found the courses too 
difficult to handle. There is no tuition fee for European students for education in Sweden, 
so there is no cost involved to sign up for courses and then change your mind. 
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A Swedish report has shown that 76% of those who took at least one ECTS credit (ap¬ 
proximately two thirds of a week of full-time studies) completed 80% of the ECTS credits 
for which they registered (HSV, 2011). This study applies to students who make a serious 
attempt to pass the course; therefore the focus is on those students who submitted at least 
one assignment during the first three weeks of the course. 

Questionnaire 

The Approaches and Study Skills Inventory for Students (ASSIST) is an evolved revi¬ 
sion of the Approaches to Studying Inventory (ASI) developed in the late 1970s (Entwistle 
& Tait, 1994). The short version of ASSIST is easily distributed and administered. It con¬ 
sists of 18 conceptually overlapping questions designed to allow students to describe how 
they go about learning and studying. 

The questionnaire used in this study is a Swedish translation of the short version of 
ASSIST. In Table 1, each question as well as its subscale is shown. The translated ques¬ 
tionnaire has passed through a cross-cultural validation process by people independent 
from the project to improve the translation and to ensure consistency in wording and 
meaning with the original version. During the development, face validity from the re¬ 
searchers’ point of view and from the students’ perspective was tested. The version of the 
Swedish translation of ASSIST used in this study has been revised to meet the require¬ 
ment of internal reliability (Ohrstedt, 2009) as tested and supported in the Finnish trans¬ 
lation (Heinstrom, 2005). 

The students score each item on a scale of 1 to 5. Responses are scored according 
to a scoring key where three subscale score variables are formed. This corresponds to 
the scoring rubric for the three study approaches. Earlier versions of the short ASSIST 
questionnaire were criticized for lack of stability compared with the full version, and the 
reliability of the full version has been questioned (Richardson, 1995). The version used 
here has been found to be reliable, and the short version correlates with the full version 
with a correlation coefficient of 0.93 for the deep and surface approaches and 0.91 for the 
strategic approach (Heinstrom, 2005). 

Since the study concerns online courses, question number 16 (surface approach, sub¬ 
scale “unrelated memorizing”) was removed (“I’m not really sure what’s important in 
lectures, so I try to get down all I can”), as it does not apply in the courses studied. This 
means that the aggregated scores for the approaches are measured on different scales; 
deep and strategic range from 6 to 30, and surface ranges from 5 to 25. 

In addition to the ASSIST survey, a question on the students’ experience of prior high¬ 
er education was added. This question was added in order to see if this exogenous factor 
correlated with, and possibly impacted, the students’ approaches and/or completion of 
the course. 

Statistical Analysis 

The results were analyzed to determine independence of samples, correlations, and 
the effects of each study approach through regression analyses. The chi-square test for 
independence was used in order to determine whether there is a significant association 
between the various approaches and completion. Kendall’s tau coefficient (t) was used 
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Table 1 

Scale Reliability Analysis of the 17 ASSIST Questions for Both Courses 

Progn mmf g at hm, atks 


Deep approach 



0.70 

0.66 

Seeking meaning 

02 

When Fm reading an article or book, I try to find out 
for myself exactly what the author means. 

0.68 

0.65 


06 

Before tackling a problem or assignment, I first try to 
work out what lies behind it. 

0.67 

0.63 

Relating ideas 

10 

When Fm working on a new topic, I try to see in my 
own mind how all the ideas fit together. 

0.63 

0.60 


15 

Ideas in course books or articles often set me off on 
long chains of thought of my own. 

0.66 

0.58 

Use of evidence 

12 

Often I find myself questioning things I hear in lec¬ 
tures or read in books. 

0.68 

0.67 


17 

When I read, I examine the details carefully to see 
how they fit in with what’s being said. 

0.64 

0-59 

Strategic approach 


0.79 

0.75 

Time manage¬ 

03 

I organize my study time carefully to make the best 

0-73 

0.71 

ment 


use of it. 




05 

I work steadily through the term or semester, rather 
than leave it all until the last minute. 

0.76 

0.71 


07 

I’m pretty good at getting down to work whenever I 
need to. 

0.78 

0-73 

Achieving 

09 

I put a lot of effort into studying because I’m deter¬ 
mined to do well. 

0-75 

0.71 


11 

I don’t find it at all difficult to motivate myself. 

0.76 

0-73 

Organized study¬ 

13 

I think I’m quite systematic and organized when it 

0.76 

0.70 

ing 


comes to revising for exams. 



Surface approach 


0-57 

0.70 

Unrelated memo¬ 

01 

I often have trouble in making sense of the things I 

0.47 

0.62 

rizing 


have to remember. 




08 

Much of what I’m studying makes little sense: it’s like 
unrelated bits and pieces. 

0.48 

0.60 

Lack of purpose 

04 

There’s not much of the work here that I find inter¬ 
esting or relevant. 

0.61 

0.70 

Fear of Failure 

14 

Often I feel I’m drowning in the sheer amount of ma¬ 
terial we’re having to cope with. 

0-47 

0.64 


18 

I often worry about whether I’ll ever be able to cope 

0-53 

0.67 


with the work properly. 


Note. The higher the value of scale alpha if an item is deleted, the less the item correlates to the 
scale as a whole. (For example, if Q04 is removed from the surface approach section, Cronbach’s 
alpha would be 0.61, compared with 0.57 if the question is kept.) A Cronbach’s alpha of 0.70 is 
usually seen as the cut-off for being acceptable, but since these values are in line with previous 
uses of the instrument, it is good enough for our objective. 
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to measure the strength of association between course completion and approaches, and 
Spearman’s rank order correlation coefficient (p) was used to measure the association 
among the three approaches. Once we established the existence of association and corre¬ 
lation, binomial logistic regression was used to evaluate the effect of each study approach 
on course completion. The three approaches act as independent variables, and pass-or- 
fail acts as the dichotomous dependent variable. Welch’s two-sample t-test was used to 
analyze whether group means differ, since the variance cannot be assumed to be equal. 

For all of the statistical analysis, a significance level of a = 0.05 is used. 

Results 

The results indicate that the internal consistency of the test is in the lower range. How¬ 
ever, the Cronbach’s alpha values for the deep (0.70 in the programming course and 0.66 
in the mathematics course), strategic (0.79/0.75) and surface approaches (0.57/0.70) re¬ 
spectively correspond to previous scores of the instrument in this Swedish translation 
(deep 0.59, strategic 0.78 and surface 0.66) as well as to the use of a Finnish transla¬ 
tion (0.66/0.67/0.63) and the original short version (0.76/0.76/0.72) (Heinstrom, 2005; 
Ohrstedt, 2009). A Cronbach’s alpha of 0.70 is usually seen as the cut-off for being ac¬ 
ceptable, but as well as being a function of item intercorrelation, it is also a function of the 
number of items in a scale and the dimensionality (Cortina, 1993). 

The Chronbach’s alpha value for the surface approach is on the lower side for the 
programming course. This is mainly due to question 04 (“There’s not much of the work 
here that I find interesting or relevant”), which has a low item-to-scale total correlation 
(see Table 1). This could mean that the question is badly articulated, but since the incon¬ 
sistency is most visible in the group who did not complete the course, it could point to a 
dissonance in this specific group. 

Sample 

There were 1,095 students enrolled in the mathematics course and 1,148 in the pro¬ 
gramming course during this period; with an overlap of 168 students, a total of 2,075 
individuals were involved in these two courses. Among these, 493 students answered the 
questionnaire. Of these 493 students, 207 were enrolled in the mathematic course and 
355 in the programming course, with an overlap of 69 students. Since the overlapping 
students represent only 14% of the respondents, the courses were analyzed separately, 
with these students being included in both groups. Students who attended both courses 
did not differ significantly from the students in general. 

The mean age among the respondents was 23.6 years in the mathematics course, with 
participants from 18 to 46, and 23.7 years in the programming course, with participants 
from 18 to 66; there were no significant age differences between the students who passed 
the course and those who did not. The nonrespondents were older than the respondents, 
with mean ages of 23.6 and 25.8, but the difference was not significant. 

The gender distribution did not differ between the two groups, with females approxi¬ 
mately 39% of the students among the respondents as well as the non-respondents. 
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Missing Values 

Three respondents answered only a few questions and were removed from the analy¬ 
sis. They are not counted in the total number of participants (Table 2). A total of 38 stu¬ 
dents did not answer one or two of the questions, and among these, five students were 
in both courses. The mean sample value for each question in each respective course was 
imputed for these missing values. 


Table 2 

Number of Participants by Gender in Subgroups 



Male 

Female 

Total 

Mathematics, pass 

45 

22 

67 

Mathematics, fail 

86 

54 

140 

Mathematics, all 

131 

76 

207 

Programming, pass 

162 

105 

267 

Programming, fail 

48 

40 

88 

Programming, all 

210 

145 

355 


Course Completion 

Completion rates differed in that respondents in the programming course performed 
better than the nonrespondents (75.2% versus 69.3%) but the opposite was true among 
students in the mathematics course (32.4% versus 36.6%), although the difference was of 
significance only in the programming course (p < 0.05). 

There was no statistically significant correlation between gender and completion; this 
was consistent for respondents and nonrespondents. In addition, there was no significant 
difference in completion between those who had a prior experience of studies in higher 
education before the course and those who had none. 

Completion Correlation and Association 

The three different approaches were separated into three categories to create frequen¬ 
cy tables, with low being the first quartile, medium the second and third quartile, and high 
the fourth quartile. Through the chi-square test, a possible association between comple¬ 
tion and the strategic approach was found in the mathematics course (p < 0.002) and 
between completion and the deep approach in the programming course (p < 0.005). No 
significant association exists in other approaches. 

Both the deep approach and the strategic approach correlate positively with comple¬ 
tion in the mathematics course, as indicated in Table 3. In both the time management 
subscale (strategic) and the relating ideas subscale (deep), there is a strong positive cor¬ 
relation with completion; the probability of passing increases as the score in each sub¬ 
scale rises (p < 0.01). The achieving subscale (strategic) is also positively correlated with 
completion (Table 4). The surface approach does not correlate with completion. 
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Table 3 

Association (t) Between Approaches and Completion, and Correlation (p) Between Approaches 



Pass, 

Deep 

Pass, 

Strategic 

Pass, 

Surface 

Deep, 

Strategic 

Deep, 

Surface 

Strategic, 

Surface 

Mathematics, pass 

- 

- 

- 

0.35** 

-0.08 

-0.11 

Mathematics, fail 

- 

- 

- 

0.24** 

-0.25** 

-0.28*** 

Mathematics, all 

0.14* 

0.16** 

-0.05 

0.28*** 

-0.20** 

-0.22** 

Programming, pass 

- 

- 

- 

0.27*** 

-0.22*** 

-0.25*** 

Programming, fail 

- 

- 

- 

0.31** 

-0.30** 

-0.09 

Programming, all 

0.11* 

-0.02 

-0.09* 

0.2 7*** 

-0.25*** 

-0.21*** 


Note, t is a value from -1 to 1. If the items were independent, the coefficient would be o; 1 is achieved if all 
pairs are concordant and -1 if they are all discordant. The critical value for p with significance level of 0.05 
is 0.195 for all groups except mathematics, pass (0.250) and programming, fail (0.217). The direction of 
the rank order relationship is determined by the + or the -. 

*p < 0.05. **p < 0.01. ***p < 0.001. 

In the programming course, completion correlates positively with a deep approach 
(Table 3), in particular the seeking meaning and relating ideas subscales (Table 4), and 
negatively with a surface approach, in particular with the fear of failure subscale. The 
pattern is visible in both subgroups, but more prominent among the students who passed 
the course. 


Table 4 

Mean Scores for Subscales 




Deep 


Strategic 


Surface 

SM 

RI 

UoE 

TM 

A 

OS 

UM 

LoP 

FoF 

Maximum score 

10.0 

10.0 

10.0 

15-0 

10.0 

5-0 

10.0 

5-0 

10.0 

Mathematics, pass 

7-7 

7-9 

7.0 

11.1 

7.8 

3-7 

4.2 

1.8 

5-3 

Mathematics, fail 

7-4 

7-1 

6.7 

10.1 

7.2 

3-5 

4 -i 

1-9 

5-5 

Mathematics, all 

7-5 

7-4 

6.8 

10.4 

7-4 

3-6 

4-1 

1.8 

5-5 

Programming, pass 

7.8 

7-5 

7.0 

10.2 

7.2 

3-6 

4.2 

1-9 

5-4 

Programming, fail 

7-3 

7.0 

6.7 

10.3 

7.2 

3-7 

4-4 

1-9 

6.0 

Programming, all 

7.6 

7-4 

6.9 

10.2 

7.2 

3-6 

4.2 

1-9 

5-6 


Note. Deep: SM = seeking meaning; RI = relating ideas; UoE = use of evidence. Strategic: TM = time man¬ 
agement; A = achieving; OS = organized studying. Surface: UM = unrelated memorizing; LoP = lack of 
purpose; FoF = fear of failure.Values shown in bold are those found to correlate with course completion. 
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Means in Approaches 

Students passing the programming course had a higher mean in the deep approach 
than those students who failed (p < 0.02) and a lower mean in the surface approach (p < 
0.04), as indicated in Table 5. They did not differ significantly in mean approach from stu¬ 
dents in the mathematics course. For the mathematics course, students who passed had a 
higher mean in the deep approach (p < 0.02) as well as in the strategic approach (p < 0.01). 


Table 5 

Mean Values with Standard Deviation and Cronbach’s Alpha to Measure Internal Consis¬ 
tency for Subgroups 




Deep 


Strategic 


Surface 


X 

s 

a 

X 

s 

a 

X 

s 

a 

Mathematics, pass 

22.6 

3-7 

0.61 

22.7 

4.6 

0-79 

11-3 

4-1 

0-75 

Mathematics, fail 

21.2 

3-9 

0.67 

20.8 

4-5 

0.72 

11.5 

3-6 

0.66 

Mathematics, all 

21.6 

3-9 

0.66 

21.4 

4.6 

0-75 

11-5 

3-7 

0.70 

Programming, pass 

22.3 

3-8 

0.68 

20.9 

4-9 

0.79 

11-5 

3-4 

0.60 

Programming, fail 

21.0 

4.2 

0.72 

21.2 

5-0 

0.80 

12.3 

3-0 

0-45 

Programming, all 

22.0 

4.0 

0.70 

21.0 

4-9 

0.79 

11.7 

3-3 

0-57 


Note. For the surface approach, the scale is 5-25, and for the deep and strategic approaches, it is 6-30. A 
Cronbach’s alpha of 0.70 is usually seen as the cut-off for being acceptable, but since these values are in 
line with previous uses of the instrument, it is good enough for our objective. 


A significant gender difference was found in the programming course, where males 
had a lower mean in the strategic approach (p < 0.001), with the time management sub¬ 
scale as the most prominent. 

Relationship Between Approaches and Probability of Completion 

The logistic regression for the mathematics course resulted in positive P for both the 
strategic and deep approaches. This indicates that the probability of completion increases 
when the score of a deep approach increases (p < 0.02) or the score of a strategic ap¬ 
proach increases (p < 0.01), as demonstrated in Figure 1. (Figures 1 and 2 show not the 
real distribution of answers but a mean distribution reached through logistic regression.) 
There is a strong correlation between the deep and strategic approaches, in that the high¬ 
er the students’ strategic approach score is, the more likely it is that they will score high 
on the deep approach. 

A positive correlation between a deep approach and a strategic approach was visible 
also in the programming course. Both approaches correlate negatively with a surface ap¬ 
proach. In the programming course the predicted probability for a student to pass in¬ 
creases when the score on the deep approach increases (p < 0.01). There is a direct, nega¬ 
tive relationship between a surface approach and the likelihood of passing the course; it 
is more probable that those with a low score on the surface approach will pass the course 
(p < 0.05), as indicated in Figure 2. 
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Deep Strategic Surface 





Figure 1. Probability of passing in the mathematics course. Additional lines denote the mean score 
for each approach (x-axis) and the throughput in the course (y-axis). 


Deep 



Strategic Surface 




Figure 2. Probability of passing in the programming course. Additional lines denote the mean 
score for each approach (x-axis) and the throughput in the course (y-axis). 


Discussion 

This study focuses on preparatory courses in programming and mathematics during 
the summer semester. The greatest difference between the two courses is that the stu¬ 
dents have studied mathematics for at least 12 years, but programming is not part of the 
core secondary school curriculum. This means that the mathematics course is intended to 
be a preparatory repetition, while the programming course is a preparatory introduction 
to its subject. The completion rate is 37% in the former and 69% in the latter, with target 
students being the same for both courses. Equivalent courses given on campus for target 
students have completion rates over 90% in both disciplines, which poses the question of 
whether the difference in completion rates is about course design rather than discipline. 

There is no difference in completion rates based on prior experience of studies in 
higher education. According to Levy (2007), there is increased probability that less ex¬ 
perienced students will drop out. This finding is not replicated in our study and may be 
related to the model of online learning or the subject area. Both courses are technical. The 
outcomes may be caused by the subject matter with its cultural and structural properties 
(Dahlberg & Marton, 1978). 
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Programming Course 

Introductory courses consist of learning facts and principles, tasks where a surface 
approach may be appropriate, particularly given time constraints in a course (Entwistle 
et al., 2000). That the opposite was demonstrated in this study is positive, since a sur¬ 
face approach is more linked to transient learning. One reason could be the design of the 
course; a coherent and congruent curriculum and the embedded assessment with fast 
feedback encourage motivated learning (van der Kleij et al., 2012). The fear of failure sub¬ 
scale was found to correlate the most with non-completion, see table 6, which could indi¬ 
cate anxiety traits in the students (Entwistle et al., 2000). Fear of failure is usually found 
to be effective in motivating high academic achievement, where the student is bound by 
a passive role and is preoccupied with thoughts of failure and intimidation by senior lec¬ 
turers (Entwistle, 1977). However, programming is a subject where failure in the sense of 
error messages from compilers is unavoidable and even something that a programmer 
needs to learn from. To be discouraged and preoccupied by this kind of failure will have a 
negative impact on the students’ learning, in terms of both process and motivation. That 
this course is short in academic credits as well as non-graded may also explain the nega¬ 
tive correlation with the subscale, as the pressure on the student is reduced compared 
with the pressure felt in longer, graded courses. 

A deep learning approach has a positive impact on self-directed lifelong learning (An¬ 
derson et al., 2011). That course completion is positively correlated to a deep approach 
is most likely due to the same factors that cause a surface approach to be negatively cor¬ 
related to it. 

No correlation between completion and a strategic approach was found, which may 
be related to the course being short and non-graded; Diseth (2003) suggests that there is 
no relationship between a strategic approach and academic achievement when there is no 
reward for achieving a high grade. 

For the students to be able to adopt more of a deep or strategic approach in their fu¬ 
ture studies, it is crucial that they have adequate prerequisite knowledge and conceptual 
understanding of the subject (Entwistle et al., 2002). The completion rate is fairly high 
for an online course, and two subscales of a deep approach, seeking meaning and relating 
ideas, are here both positively correlated with completion. But the lack of relationship 
between completion and a strategic approach could prove to be problematic for the stu¬ 
dents in the long run, since a deep or strategic approach is more positively linked to high 
academic achievement and a more active engagement and interest in the subject (Diseth 
& Martinsen, 2003; Entwistle et al., 2000). 

Mathematics Course 

Students in the mathematics course should already know the content and need only to 
refresh their knowledge. A surface approach did not significantly affect the probability of 
completing the course. A combined deep/strategic approach would be preferable, because 
the probability of completion increases as the score in the deep or strategic approach in¬ 
creases. Since that combination is encouraged, this indicates either that the mathematics 
learned in secondary school gives these students sufficient knowledge about the subject, 
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or that the current design of the course or the students’ combination of intent and motiva¬ 
tion in studying the course is sufficient (Dahlgren & Marton, 1978; Entwistle et al., 2002; 
Anderson et al., 2011. 

The subscales that significantly correlate positively with completion were relating 
ideas (deep approach) and time management and achieving (strategic approach), all of 
which could be related to the students’ intent and motivation, see table 6. 


Table 6 

The Effect of Various Approaches on Completion 



Mathematics 

Programming 

Deep approach 

Positive * 

Positive ** 

Seeking meaning 

- 

Positive ** 

Relating ideas 

Positive ** 

Positive * 

Use of evidence 

- 

- 

Strategic approach 

Positive ** 

None 

Time management 

Positive ** 

- 

Achieving 

Positive * 

- 

Organized studying 

- 

- 

Surface approach 

None 

Negative * 

Unrelated memorizing 

- 

- 

Lack of purpose 

- 

- 

Fear of failure 

- 

Negative * 


*p < 0.05. **p < 0.01. ***p < 0.001. 

Based on the results from this study, the mathematics course intentions, its outline 
and the students’ approaches are aligned, so the course convey/mediate what they are 
supposed or expected to. The fixed linear order through the course may encourage the 
students to continue. Students who follow the pre-made study plan while studying disci¬ 
plined are more likely to complete the course. 

Conclusion 

This study used ASSIST to explore completion rates in students in two online prepara¬ 
tory courses in mathematics and programming. Results show that ASSIST has a satisfac¬ 
tory internal consistency and reliability to be used with online courses. The translated 
version could therefore be used as an instrument to evaluate the effect of course design on 
students’ approaches to learning and studying and on their completion rate. 

Since the target students are the same for the two courses, the aim of the study was to 
investigate whether ASSIST could be used to explain the difference in completion rates 
between the courses. On this aspect, the data showed no correlations. However, in the 
mathematics course, students demonstrating a deep or strategic approach were more 
likely to complete the course. In the programming course, a deep approach was also posi¬ 
tively correlated with completion, while students demonstrating a surface approach were 
less likely to complete. The relating ideas subscale of the deep approach was positively 
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correlated to completion in both courses. This indicates that students who are learning- 
oriented do well, and even though the courses are both given online, they reward students 
who adopt a deep approach to learning. 

This is in line with earlier research that shows that students who perceive the assess¬ 
ment as memorization and the workload as high are more likely to adopt a surface ap¬ 
proach to learning (Trigwell, Prosser, & Waterhouse, 1999). The self-paced aspect of the 
courses should prevent students from perceiving the workload as high, and there are 
aspects in the assessment that are meant to oppose memorization. A surface approach 
to learning has been found to be linked to a passive information transmission from the 
teachers’ side in on-campus education (Trigwell et al., 1999); linking conceptual ques¬ 
tions in the course material has been a way to try to defeat this passivity. 

We therefore argue that ASSIST can be used to evaluate online course design with re¬ 
spect to which approaches to learning and instructional strategies the courses encourage 
and discourage, ultimately affecting both knowledge gains and course completion. 

Future Work 

Based on these results, there will be further research with course designers and in¬ 
structors to determine effective modifications in the courses to encourage a more deep or 
strategic approach to learning (Diseth & Martinsen, 2003; Newbie & Hejka, 1991). The 
modifications will be made with the existing course designs and the results of the study 
as well as the limitations of technical systems and financial resources in mind. It will be 
done with the hope of increasing thefcompletion rate and, ultimately, better preparing the 
students for their future education. 
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